Formation and dynamics of modules in a dual-tasking multi-layer feed-forward neural network

We study a feed-forward neural network for two independent function approximation tasks. Upon training, two modules are automatically formed in the hidden layers, each handling one of the tasks predominantly. We demonstrate that the sizes of the modules can be dynamically driven by varying the complexities of the tasks. The network serves as a simple example of an artiicial neural network with an adaptable modular structure. This study is motivated by similar dynamical nature of modules in animal brains.